CLAIMS 



What is claimed is: 

1. A method for generating a depth map used to render objects 
comprising: 

establishing a virtual camera using camera data of a tracked camera which 
defines a viewpoint for a synthetic scene; 

setting the field of view of the virtual camera to the field of view of the tracked 
camera; 

positioning a synthetic tracked object in the scene according to position 
information of the tracked object; and 

extracting depth information of the positioned synthetic tracked object to 
generate depth map. 

2. The method as set forth in claim 1, wherein the synthetic scene comprises 
a three dimensional model of an environment. 

3. The method as set forth in claim 1, wherein extracting further comprises 
extracting depth information of the synthetic scene. 

4. The method as set forth in claim 1, wherein the environment comprises a 
race track. 

5. The method as set forth in claim 1, wherein establishing further comprises 
re-positioning the virtual camera as the tracked camera changes. 

6. The method as set forth in claim 1, wherein the camera data is generated 
by a live camera focused on at least part of a live scene corresponding to the synthetic 
scene. 
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, 7. The method as set forth in claim 1, further comprising animating the 
synthetic tracked object according to the position and orientation information. 

8. The method as set forth in claim 6, further comprising combining the 
synthetic tracked object with the live scene using depth information stored in the depth 
map. 

9. The method as set forth in claim 8, further comprising using image 
segmentation to ahgn the depth map in accordance with real edges in the live scene. 

10. The method as set forth in claim 1 , wherein camera data is recorded from 
a camera focused on at least part of a scene corresponding to the synthetic scene, the 
method further comprising combining the synthetic tracked object with video using 
depth information stored in the depth map. 

11. The method as set forth in claim 1, wherein the depth map is further 
refined by distorting grid coordinates of the depth map based upon characteristics of the 
tracked camera which defines the field of view. 

12. A system for generating a depth map used to render objects comprising: 
a tracked camera configured to define a viewpoint for a synthetic scene; 

a virtual camera using camera data of the tracked camera, the field of view of the 
virtual camera set to the field of view of the tracked camera; 

a synthetic tracked object in the scene positioned according to position 
information of the tracked object, wherein depth information of the tracked object 
provides information to generate a depth map. 

13. The system as set forth in claim 12, wherein the synthetic scene comprises 
a three dimensional model of an environment. 

14. The system as set forth in claim 12, wherein the environment comprises a 
race track. 
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15. The system as set forth in claim 12, wherein depth information of the 
synthetic scene further provides information to generate the depth map. 

16. The system as set forth in claim 12, wherein the virtual camera is re- 
positioned to track camera data changes. 

17. The system as set forth in claim 12, wherein the tracked camera comprises 
a live camera and the camera data is generated by the tracked camera focused on at least 
part of a live scene corresponding to the sjmthetic scene. 

18. The system as set forth in claim 12, further comprising an animation unit 
to animate the model of the tracked object according to the position and orientation 
information. 

19. The system as set forth in claim 17, further comprising a rendering unit 
configured to generate the synthetic object with the live scene using depth information 
stored in the depth map. 

20. The system as set forth in claim 19, wherein the rendering unit is further 
configured to use image segmentation to define the depth map in accordance with real 
edges in the live scene. 

21 . The system as set forth in claim 20, wherein the rendering unit distorts 
grid coordinates of the depth map based upon characteristics of the tracked camera which 
defines the field of view. 

22. A system for generating a depth map used to render synthetic objects 
comprising: 

a tracked camera configured to define a viewpoint for a synthetic scene; 
a processor configured to generate a virtual camera using camera data of the 
tracked camera, the field of view of the virtual camera set to the field of view of the 
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tracked camera and a synthetic tracked object in the scene positioned according to 
position information of the tracked object; and 

a depth map of the positioned tracked object. 

23. The system as set forth in claim 22, wherein the synthetic scene comprises 
a three dimensional model of an environment. 

24. The system as set forth in claim 22, wherein depth information of the 
synthetic scene further provides information to generate the depth map. 

25. The system as set forth in claim 22, wherein the virtual camera is re- 
positioned to track camera data changes. 

26. The system as set forth in claim 22, wherein the tracked camera comprises 
a live camera coupled to the processor and the camera data is generated by the tracked 
camera focused on at least part of a live scene corresponding to the synthetic scene. 

27. The system as set forth in claim 22,wherein the processor is further 
configured to animate the model of the tracked object according to the position and 
orientation information. 

28. The system as set forth in claim 26, wherein the processor is further 
configured to generate the synthetic model with the live scene using depth information 
stored in the depth map. 

29. The system as set forth in claim 28, wherein the processor is further 
configured to use image segmentation to define the depth map in accordance with real 
edges in the live scene. 

30. The system as set forth in claim 29, wherein the processor is further 
configured to distort grid coordinates of the depth map based upon characteristics of the 
tracked camera which defines the field of view. 
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31. A computer readable medium comprising instructions, which when 
executed by a processing system perform a method for generating a depth map used to 
render objects comprising: 

establishing a virtual camera using camera data of a tracked camera which 
defines a viewpoint for a synthetic scene; 

setting the field of view of the virtual camera to the field of view of the tracked 
camera; 

positioning a synthetic tracked object in the scene according to position 
information of the tracked object; and 

extracting depth information of the positioned synthetic tracked object to 
generate a depth map. 

32. The computer readable medium as set forth in claim 31, wherein the 
synthetic scene comprises a three dimensional model of an environment. 

33. The computer readable medium as set forth in claim 3 1 , wherein 
establishing further comprises re-positioning the virtual camera to track camera data 
changes. 

34. The computer readable medium as set forth in claim 3 1 , wherein depth 
information of the synthetic scene is further used to generate the depth map. 

35. The computer readable medium as set forth in claim 3 1 , wherein the 
camera data is generated by a live camera focused on at least part of a live scene 
corresponding to the synthetic scene. 

36. The computer readable medium as set forth in claim 31, further 
comprising animating the model of the tracked object according to the position and 
orientation information. 
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. 37. The computer readable medium as set forth in claim 35, further 
comprising combining the synthetic object with the live scene using depth information 
stored in the depth map. 

38. The computer readable medium as set forth in claim 37, further 
comprising using image segmentation to define the depth map in accordance with real 
edges in the live scene. 

39. The computer readable medium as set forth in claim 3 1 , further 
comprising distorting grid coordinates of the depth map based upon characteristics of the 
tracked camera which defines the field of view. 
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